Why Microsoft.Extensions.AI exists

Every AI provider ships its own .NET SDK, and they all disagree. OpenAI has ChatClient. Azure has AzureOpenAIClient. Ollama has OllamaApiClient. Amazon has AmazonBedrockRuntimeClient. They have different method names, different message shapes, different streaming types and different ways of describing a tool call. Write your application against one of them and you have married that provider — moving to another means rewriting every call site.

Series explained

A nine-part written series on building AI features into .NET applications with Microsoft.Extensions.AI — the same abstraction stack that powers chat, embeddings, retrieval, image generation, on-device inference and cloud inference behind one set of interfaces.

Every article builds on the previous one. Together they produce a single .NET MAUI application that:

• streams chat responses from a local LLM running on your own machine,
• remembers the conversation and follows system instructions,
• calls your own C# methods as tools,
• ingests PDFs, vectorises them and answers questions from their content (RAG),
• generates images from a prompt,
• switches to Apple Intelligence on-device when the hardware supports it,
• and falls back to Microsoft Foundry or Amazon Bedrock in the cloud,

all without a single if (provider == …) in the application code.

The full source code of this series is available on GitHub.

The series

Prerequisites

• .NET 10 SDK
• A machine with ~8 GB of free RAM for local models (16 GB is comfortable)
• Ollama for articles 1–7
• Optional for article 8: a physical iOS 26 / macOS 26 device with Apple Intelligence enabled
• Optional for article 9: a Microsoft Foundry deployment or an AWS account with Bedrock model access

Package versions used throughout

The series pins the versions below via central package management (Directory.Packages.props). Everything is current as of July 2026.

<PackageVersion Include="Microsoft.Extensions.AI" Version="10.8.1" />
<PackageVersion Include="Microsoft.Extensions.AI.Abstractions" Version="10.8.1" />
<PackageVersion Include="Microsoft.Extensions.AI.OpenAI" Version="10.8.1" />
<PackageVersion Include="Microsoft.Extensions.AI.Evaluation.Quality" Version="10.8.0" />
<PackageVersion Include="Microsoft.Extensions.AI.Evaluation.NLP" Version="10.8.0-preview.1.26364.2" />
<PackageVersion Include="Microsoft.Extensions.VectorData.Abstractions" Version="10.8.0" />
<PackageVersion Include="Microsoft.SemanticKernel.Connectors.InMemory" Version="1.74.0-preview" />
<PackageVersion Include="Microsoft.SemanticKernel.Connectors.SqliteVec" Version="1.74.0-preview" />
<PackageVersion Include="Microsoft.Maui.Essentials.Ai" Version="10.0.51-preview.1.26168.3" />
<PackageVersion Include="OllamaSharp" Version="5.4.27" />
<PackageVersion Include="Azure.AI.OpenAI" Version="2.9.0-beta.1" />
<PackageVersion Include="AWSSDK.BedrockRuntime" Version="4.0.13" />
<PackageVersion Include="AWSSDK.Extensions.Bedrock.MEAI" Version="4.0.6.2" />
<PackageVersion Include="PdfPig" Version="0.1.14" />

Keep the AI packages on one version line. Microsoft.Extensions.AI, .Abstractions and .OpenAI must all move together, and third-party providers like OllamaSharp pull in .Abstractions transitively — if the central pin lags behind what a provider requires, NuGet raises NU1109 (package downgrade). Bump them as a set.

Microsoft.Extensions.AI overview

Microsoft.Extensions.AI is Microsoft’s answer to that problem, and it is the same answer they gave for logging (ILogger), configuration (IConfiguration) and HTTP (HttpClient + IHttpClientFactory): a small set of abstractions in the box, and providers that plug in behind them. You program against the interface; the provider is a registration detail.

There are four core abstractions:

Get comfortable with IChatClient and the rest feel familiar — they follow the same design.

The one interface you have to know

IChatClient has two methods that matter:

• GetResponseAsync — send messages, await the whole reply as a ChatResponse.
• GetStreamingResponseAsync — send messages, receive an IAsyncEnumerable<ChatResponseUpdate> that yields fragments as the model produces them. This is what gives you the ChatGPT-style “typewriter” effect.

That is the entire surface you need to build a chatbot.

Set up a local model first

You do not need a cloud account or an API key to follow along. Articles 1–7 run against Ollama, which serves open models on your own machine over http://localhost:11434.

# once, after installing Ollama
ollama serve             # start the local server (often already running)
ollama pull qwen3:1.7b   # a small, tool-capable chat model

qwen3:1.7b is a good default: roughly 1.4 GB on disk, it supports tool calling (which we need in article 3), and it runs on a laptop without a dedicated GPU.

A word on size. Plain ollama pull qwen3 gives you the 8-billion-parameter build — about 5 GB of weights that must all sit in memory. If you have a decent discrete GPU, use it and enjoy the better answers. If you are on integrated graphics, Ollama falls back to the CPU and that model becomes painful: on an Intel Iris Xe laptop it generates around 4 tokens per second, which turns the one-sentence example below into a two-minute wait. Start with 1.7b, and size up later once the code works.

Hello, IChatClient

Here is the smallest possible program. One package reference (Microsoft.Extensions.AI) plus a provider (OllamaSharp):

using Microsoft.Extensions.AI;
using OllamaSharp;

const string modelId = "qwen3:1.7b";
var endpoint = new Uri("http://127.0.0.1:11434");

// OllamaApiClient implements IChatClient directly. Swapping providers means
// changing this one line -- nothing below it moves.
using IChatClient client = new OllamaApiClient(endpoint, modelId);

// Buffered: wait for the whole answer.
ChatResponse response = await client.GetResponseAsync("Explain async/await in one sentence.");
Console.WriteLine(response.Text);

The variable is typed as IChatClient, not OllamaApiClient. That is deliberate and it is the whole point: everything after this line is written against the abstraction, so when article 9 swaps Ollama for Azure or Bedrock, this is the only line that changes.

Streaming

await foreach (ChatResponseUpdate update in client.GetStreamingResponseAsync(
      "Tell me a short story about a developer attending .NET Conf for the first time."))
{
    Console.Write(update.Text);
}

Each ChatResponseUpdate is a small piece of the answer. You render it the instant it arrives instead of staring at a spinner for ten seconds — the difference between an app that feels alive and one that feels broken.

Usage metadata comes along for free

When the provider reports token counts, they ride on the response:

if (response.Usage is { } usage)
{
    Console.WriteLine($"Input tokens:  {usage.InputTokenCount}");
    Console.WriteLine($"Output tokens: {usage.OutputTokenCount}");
}

Same property on every provider — you write your cost logging once.

Run it

cd code
dotnet run --project 01.HelloChatClient

You will see the buffered answer print at once, then the story stream in word by word.

When it does not work

Three failure modes account for almost every “it just sits there” report on this first sample. They look similar from the console and have nothing to do with each other.

500 (Internal Server Error) on the very first call

System.Net.Http.HttpRequestException:
Response status code does not indicate success: 500 (Internal Server Error).

OllamaSharp surfaces any non-2xx response as an HttpRequestException and throws away the body — which is where Ollama put the actual reason. Ask the server directly and it tells you:

curl -X POST http://127.0.0.1:11434/api/chat \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3:1.7b","messages":[{"role":"user","content":"hi"}],"stream":false}'

{"error":"llama-server process has terminated: exit status 1:
ggml_backend_cpu_buffer_type_alloc_buffer: failed to allocate buffer of size 3312451584
error loading model: unable to allocate CPU_REPACK buffer"}

That is not your code. Ollama could not fit the model in RAM. The 8B build needs about 5 GB free; a browser and a couple of IDE windows will happily deny you that. Close things, or use the smaller model. ollama ps shows what is loaded and whether size_vram is 0 — if it is, you are running on the CPU.

Worth wrapping the call so the reason is not invisible:

try
{
    ChatResponse response = await client.GetResponseAsync("Explain async/await in one sentence.");
    Console.WriteLine(response.Text);
}
catch (HttpRequestException ex)
{
    Console.Error.WriteLine($"Ollama rejected the request: {ex.Message}");
    Console.Error.WriteLine("Is `ollama serve` running, and is the model pulled?");
}

The console stays blank, then everything appears at once

qwen3 is a reasoning model. Before it writes a word of the answer it emits a long internal monologue, and Ollama returns that in a separate thinking field rather than as message content. On the 8B model a one-sentence answer measured 437 tokens — of which roughly 390 were thinking and about 50 were the reply.

This matters because of how Microsoft.Extensions.AI models content. ChatResponseUpdate.Text concatenates TextContent only. Reasoning arrives as TextReasoningContent, so this loop prints nothing at all during the thinking phase:

await foreach (ChatResponseUpdate update in client.GetStreamingResponseAsync(prompt))
{
    Console.Write(update.Text);   // silent until the model stops thinking
}

Walk update.Contents instead and you can see the model working:

await foreach (ChatResponseUpdate update in client.GetStreamingResponseAsync(prompt))
{
    foreach (AIContent content in update.Contents)
    {
        switch (content)
        {
            case TextReasoningContent reasoning:
                Console.ForegroundColor = ConsoleColor.DarkGray;
                Console.Write(reasoning.Text);
                Console.ResetColor();
                break;
            case TextContent text:
                Console.Write(text.Text);
                break;
        }
    }
}

The distinction between TextContent and TextReasoningContent is the first place the abstraction earns its keep: the provider decides what a “thinking” token is, and your code just pattern-matches on content types.

Nothing you change makes any difference

If you are debugging in Visual Studio, an unhandled exception pauses the process rather than ending it. That paused process keeps holding bin\Debug\net10.0\HelloChatClient.exe, so the next build fails before it ever runs:

error MSB3027: Could not copy "obj\Debug\net10.0\apphost.exe" to
"bin\Debug\net10.0\HelloChatClient.exe". Exceeded retry count of 10.
The file is locked by: "HelloChatClient (71492)"

You fix the real problem, hit run, and observe no change — because the code never rebuilt. Press Shift+F5 (Debug → Stop Debugging) to end the stale session before re-running.

What you just proved

You wrote a working AI program with no cloud dependency, no API key and no provider-specific types past the constructor. The mental model for the rest of the series is set:

The provider is behind the interface. Your code is in front of it.

Every article widens that front — history, tools, retrieval, images — without ever reaching back through the interface to a specific provider.

Next: Your first IChatClient — turning this one-shot call into a real chat with dependency injection, middleware, memory and system instructions.